{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.4 TensorFlow模型持久化\n",
    "在5.2.1节中给出的样例代码在训练完成后就直接退出了，并没有将训练得到的模型保存下来方便下次直接使用。**为了让训练结果可以复用，需要将得到的神经网络模型持久化。**\n",
    "\n",
    "5.4.1节将介绍通过TensorFlow程序来持久化一个训练好的模型，并从持久化之后的模型文件中还原被保存的模型；5.4.2节将介绍TensorFlow持久化的工作原理和持久化之后文件中的数据格式。\n",
    "\n",
    "### 5.4.1 持久化代码实现\n",
    "**1. 保存模型**\n",
    "\n",
    "**TensorFlow提供了一个非常简单的API来保存和还原一个神经网络模型，这个API就是tf.train.Saver类**，如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "# 声明两个变量并计算他们的和\n",
    "v1 = tf.Variable(tf.constant(1.0, shape=[1]), name='v1')\n",
    "v2 = tf.Variable(tf.constant(1.0, shape=[1]), name='v2')\n",
    "result = v1 + v2\n",
    "\n",
    "init_op = tf.global_variables_initializer()\n",
    "# 声明tf.train.Saver类用于保存模型\n",
    "saver = tf.train.Saver()\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    sess.run(init_op)\n",
    "    saver.save(sess, \"Saved_model/model.ckpt\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上述代码中，`saver.save` 函数将TensorFlow模型保存到了*Saved_model/model.ckpt*文件中。上述程序虽然只指定了一个文件路径，但是在这个目录下会出现三个文件，这是因为TensorFlow会将计算图的结构和图上参数取值分开保存，这几个文件分别为（这一部分参考了5.4.2节）：\n",
    "\n",
    "**a. model.ckpt.meta**，它保存了TensorFlow计算图的结构，可以简单理解为神经网络的网络结构；\n",
    "\n",
    "**b. model.ckpt**，它保存了模型中所有变量的取值，实际上分为两个文件：model.ckpt.index 和 model.ckpt.data-\\*\\*\\*\\*\\*-of-\\*\\*\\*\\*\\*，其中后者是通过SStable格式存储的，可以大致理解为一个（key, value）的列表。TensorFlow提供了`tf.train.NewCheckpointReader`类来查看保存的变量信息，如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "v1 [1]\n",
      "v2 [1]\n",
      "Value for variable v1 is  [1.]\n"
     ]
    }
   ],
   "source": [
    "# 该类可以读取checkpoint文件中保存的所有变量，注意后面的.data和.index可以省去\n",
    "reader = tf.train.NewCheckpointReader('Saved_model/model.ckpt')\n",
    "\n",
    "# 获取所有变量列表，这个是一个从变量名到变量维度的字典\n",
    "global_varibales = reader.get_variable_to_shape_map()\n",
    "for variable_name in global_varibales:\n",
    "    # variable_name为变量名称，global_variables[variable_name]为变量的维度\n",
    "    print(variable_name, global_varibales[variable_name])\n",
    "    \n",
    "# 获取名为v1的变量的取值\n",
    "print(\"Value for variable v1 is \", reader.get_tensor(\"v1\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**c. checkpoint**，这个文件是 `tf.train.Saver` 类自动生成生成且自动维护的。在checkpoint文件中维护了一个由 `tf.train.Saver` 类持久化的所有TensorFlow模型文件的文件名。当某个保存的TensorFlow模型文件被删除时，这个模型对应的文件名也会从checkpoint文件中删除。checkpoint中内容的格式为 CheckpointState Protocol Buffer，下面给出了 CheckpointState 类型的定义：\n",
    "\n",
    "`message CheckpointState {\n",
    "  string model_checkpoint_path = 1;\n",
    "  repeated string all_model_checkpoint_paths = 2;\n",
    "}`\n",
    "\n",
    "- model_checkpoint_path 属性保存了最新的TensorFlow模型文件的文件名；\n",
    "- all_model_checkpoint_paths 属性列出了当前还没有被删除的所有TensorFlow模型文件的文件名。\n",
    "\n",
    "对于本例：\n",
    "\n",
    " `model_checkpoint_path: \"model.ckpt\"\n",
    " all_model_checkpoint_paths: \"model.ckpt\"\n",
    "`\n",
    "\n",
    "**2. 加载模型**\n",
    "\n",
    "下面的代码显示了如何**加载已经保存的模型**（建议restart the kernel，然后从这个cell运行，下同）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from Saved_model/model.ckpt\n",
      "[2.]\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "# 使用和保存模型代码中一样的方式来声明变量\n",
    "v1 = tf.Variable(tf.constant(1.0, shape=[1]), name='v1')\n",
    "v2 = tf.Variable(tf.constant(1.0, shape=[1]), name='v2')\n",
    "result = v1 + v2\n",
    "\n",
    "saver = tf.train.Saver()\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    # 加载已经保存的模型，并通过已保存的模型中变量来计算加法\n",
    "    saver.restore(sess, \"Saved_model/model.ckpt\")\n",
    "    print(sess.run(result))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这段代码基本和保存模型的代码是一样的，也是先定义了TensorFlow计算图上的所有运算，并声明了一个 `tf.train.Saver` 类。**唯一不同的是在加载模型的代码中没有运行变量的初始化过程，而是将变量的值通过已经保存的模型加载进来。**\n",
    "\n",
    "**如果不希望重复定义图上的计算，也可以直接加载已经持久化的图**，如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from Saved_model/model.ckpt\n",
      "[2.]\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "saver = tf.train.import_meta_graph(\"Saved_model/model.ckpt.meta\")\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    saver.restore(sess, \"Saved_model/model.ckpt\")\n",
    "    # 通过张量名称来获取张量\n",
    "    print(sess.run(tf.get_default_graph().get_tensor_by_name(\"add:0\")))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**3. 保存或加载部分变量**\n",
    "\n",
    "上面给出的程序，默认都是保存或加载了TensorFlow计算图上定义的所有变量，但有时可能只需要保存或加载部分变量，如一个训练好神经网络的前五层。**为了保存或者加载部分变量，在声明 `tf.train.Saver` 类时可以提供一个列表来指定需要保存或加载的变量**。\n",
    "\n",
    "如在加载模型的代码中将 `saver = tf.train.Saver()` 改为 `saver = tf.train.Saver([v1])`，可以看到就会报错`FailedPreconditionError: Attempting to use uninitialized value v2`，如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from Saved_model/model.ckpt\n"
     ]
    },
    {
     "ename": "FailedPreconditionError",
     "evalue": "Attempting to use uninitialized value v2\n\t [[{{node v2/read}} = Identity[T=DT_FLOAT, _device=\"/job:localhost/replica:0/task:0/device:GPU:0\"](v2)]]\n\t [[{{node add/_3}} = _Recv[client_terminated=false, recv_device=\"/job:localhost/replica:0/task:0/device:CPU:0\", send_device=\"/job:localhost/replica:0/task:0/device:GPU:0\", send_device_incarnation=1, tensor_name=\"edge_9_add\", tensor_type=DT_FLOAT, _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"]()]]\n\nCaused by op 'v2/read', defined at:\n  File \"d:\\python3\\Lib\\runpy.py\", line 193, in _run_module_as_main\n    \"__main__\", mod_spec)\n  File \"d:\\python3\\Lib\\runpy.py\", line 85, in _run_code\n    exec(code, run_globals)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel_launcher.py\", line 16, in <module>\n    app.launch_new_instance()\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\traitlets\\config\\application.py\", line 658, in launch_instance\n    app.start()\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel\\kernelapp.py\", line 486, in start\n    self.io_loop.start()\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tornado\\platform\\asyncio.py\", line 112, in start\n    self.asyncio_loop.run_forever()\n  File \"d:\\python3\\Lib\\asyncio\\base_events.py\", line 421, in run_forever\n    self._run_once()\n  File \"d:\\python3\\Lib\\asyncio\\base_events.py\", line 1431, in _run_once\n    handle._run()\n  File \"d:\\python3\\Lib\\asyncio\\events.py\", line 145, in _run\n    self._callback(*self._args)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tornado\\platform\\asyncio.py\", line 102, in _handle_events\n    handler_func(fileobj, events)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tornado\\stack_context.py\", line 276, in null_wrapper\n    return fn(*args, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\zmq\\eventloop\\zmqstream.py\", line 450, in _handle_events\n    self._handle_recv()\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\zmq\\eventloop\\zmqstream.py\", line 480, in _handle_recv\n    self._run_callback(callback, msg)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\zmq\\eventloop\\zmqstream.py\", line 432, in _run_callback\n    callback(*args, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tornado\\stack_context.py\", line 276, in null_wrapper\n    return fn(*args, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel\\kernelbase.py\", line 283, in dispatcher\n    return self.dispatch_shell(stream, msg)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel\\kernelbase.py\", line 233, in dispatch_shell\n    handler(stream, idents, msg)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel\\kernelbase.py\", line 399, in execute_request\n    user_expressions, allow_stdin)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel\\ipkernel.py\", line 208, in do_execute\n    res = shell.run_cell(code, store_history=store_history, silent=silent)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel\\zmqshell.py\", line 537, in run_cell\n    return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2728, in run_cell\n    interactivity=interactivity, compiler=compiler, result=result)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2850, in run_ast_nodes\n    if self.run_code(code, result):\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2910, in run_code\n    exec(code_obj, self.user_global_ns, self.user_ns)\n  File \"<ipython-input-1-9ee245d85979>\", line 5, in <module>\n    v2 = tf.Variable(tf.constant(1.0, shape=[1]), name='v2')\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\ops\\variables.py\", line 145, in __call__\n    return cls._variable_call(*args, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\ops\\variables.py\", line 141, in _variable_call\n    aggregation=aggregation)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\ops\\variables.py\", line 120, in <lambda>\n    previous_getter = lambda **kwargs: default_variable_creator(None, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\ops\\variable_scope.py\", line 2441, in default_variable_creator\n    expected_shape=expected_shape, import_scope=import_scope)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\ops\\variables.py\", line 147, in __call__\n    return super(VariableMetaclass, cls).__call__(*args, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\ops\\variables.py\", line 1104, in __init__\n    constraint=constraint)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\ops\\variables.py\", line 1266, in _init_from_args\n    self._snapshot = array_ops.identity(self._variable, name=\"read\")\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\ops\\array_ops.py\", line 81, in identity\n    return gen_array_ops.identity(input, name=name)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\ops\\gen_array_ops.py\", line 3994, in identity\n    \"Identity\", input=input, name=name)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\framework\\op_def_library.py\", line 787, in _apply_op_helper\n    op_def=op_def)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\util\\deprecation.py\", line 488, in new_func\n    return func(*args, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\framework\\ops.py\", line 3272, in create_op\n    op_def=op_def)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\framework\\ops.py\", line 1768, in __init__\n    self._traceback = tf_stack.extract_stack()\n\nFailedPreconditionError (see above for traceback): Attempting to use uninitialized value v2\n\t [[{{node v2/read}} = Identity[T=DT_FLOAT, _device=\"/job:localhost/replica:0/task:0/device:GPU:0\"](v2)]]\n\t [[{{node add/_3}} = _Recv[client_terminated=false, recv_device=\"/job:localhost/replica:0/task:0/device:CPU:0\", send_device=\"/job:localhost/replica:0/task:0/device:GPU:0\", send_device_incarnation=1, tensor_name=\"edge_9_add\", tensor_type=DT_FLOAT, _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"]()]]\n",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mFailedPreconditionError\u001b[0m                   Traceback (most recent call last)",
      "\u001b[1;32md:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36m_do_call\u001b[1;34m(self, fn, *args)\u001b[0m\n\u001b[0;32m   1291\u001b[0m     \u001b[1;32mtry\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1292\u001b[1;33m       \u001b[1;32mreturn\u001b[0m \u001b[0mfn\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m*\u001b[0m\u001b[0margs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   1293\u001b[0m     \u001b[1;32mexcept\u001b[0m \u001b[0merrors\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mOpError\u001b[0m \u001b[1;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32md:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36m_run_fn\u001b[1;34m(feed_dict, fetch_list, target_list, options, run_metadata)\u001b[0m\n\u001b[0;32m   1276\u001b[0m       return self._call_tf_sessionrun(\n\u001b[1;32m-> 1277\u001b[1;33m           options, feed_dict, fetch_list, target_list, run_metadata)\n\u001b[0m\u001b[0;32m   1278\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32md:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36m_call_tf_sessionrun\u001b[1;34m(self, options, feed_dict, fetch_list, target_list, run_metadata)\u001b[0m\n\u001b[0;32m   1366\u001b[0m         \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_session\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0moptions\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfeed_dict\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfetch_list\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtarget_list\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1367\u001b[1;33m         run_metadata)\n\u001b[0m\u001b[0;32m   1368\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mFailedPreconditionError\u001b[0m: Attempting to use uninitialized value v2\n\t [[{{node v2/read}} = Identity[T=DT_FLOAT, _device=\"/job:localhost/replica:0/task:0/device:GPU:0\"](v2)]]\n\t [[{{node add/_3}} = _Recv[client_terminated=false, recv_device=\"/job:localhost/replica:0/task:0/device:CPU:0\", send_device=\"/job:localhost/replica:0/task:0/device:GPU:0\", send_device_incarnation=1, tensor_name=\"edge_9_add\", tensor_type=DT_FLOAT, _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"]()]]",
      "\nDuring handling of the above exception, another exception occurred:\n",
      "\u001b[1;31mFailedPreconditionError\u001b[0m                   Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-1-9ee245d85979>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[0;32m     11\u001b[0m     \u001b[1;31m# 加载已经保存的模型，并通过已保存的模型中变量来计算加法\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     12\u001b[0m     \u001b[0msaver\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mrestore\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0msess\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m\"Saved_model/model.ckpt\"\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 13\u001b[1;33m     \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0msess\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mrun\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mresult\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;32md:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36mrun\u001b[1;34m(self, fetches, feed_dict, options, run_metadata)\u001b[0m\n\u001b[0;32m    885\u001b[0m     \u001b[1;32mtry\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    886\u001b[0m       result = self._run(None, fetches, feed_dict, options_ptr,\n\u001b[1;32m--> 887\u001b[1;33m                          run_metadata_ptr)\n\u001b[0m\u001b[0;32m    888\u001b[0m       \u001b[1;32mif\u001b[0m \u001b[0mrun_metadata\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    889\u001b[0m         \u001b[0mproto_data\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mtf_session\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mTF_GetBuffer\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mrun_metadata_ptr\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32md:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36m_run\u001b[1;34m(self, handle, fetches, feed_dict, options, run_metadata)\u001b[0m\n\u001b[0;32m   1108\u001b[0m     \u001b[1;32mif\u001b[0m \u001b[0mfinal_fetches\u001b[0m \u001b[1;32mor\u001b[0m \u001b[0mfinal_targets\u001b[0m \u001b[1;32mor\u001b[0m \u001b[1;33m(\u001b[0m\u001b[0mhandle\u001b[0m \u001b[1;32mand\u001b[0m \u001b[0mfeed_dict_tensor\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1109\u001b[0m       results = self._do_run(handle, final_targets, final_fetches,\n\u001b[1;32m-> 1110\u001b[1;33m                              feed_dict_tensor, options, run_metadata)\n\u001b[0m\u001b[0;32m   1111\u001b[0m     \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1112\u001b[0m       \u001b[0mresults\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;33m[\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32md:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36m_do_run\u001b[1;34m(self, handle, target_list, fetch_list, feed_dict, options, run_metadata)\u001b[0m\n\u001b[0;32m   1284\u001b[0m     \u001b[1;32mif\u001b[0m \u001b[0mhandle\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1285\u001b[0m       return self._do_call(_run_fn, feeds, fetches, targets, options,\n\u001b[1;32m-> 1286\u001b[1;33m                            run_metadata)\n\u001b[0m\u001b[0;32m   1287\u001b[0m     \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1288\u001b[0m       \u001b[1;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_do_call\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0m_prun_fn\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mhandle\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfeeds\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfetches\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32md:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36m_do_call\u001b[1;34m(self, fn, *args)\u001b[0m\n\u001b[0;32m   1306\u001b[0m           self._config.experimental.client_handles_error_formatting):\n\u001b[0;32m   1307\u001b[0m         \u001b[0mmessage\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0merror_interpolation\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0minterpolate\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mmessage\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_graph\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1308\u001b[1;33m       \u001b[1;32mraise\u001b[0m \u001b[0mtype\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0me\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnode_def\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mop\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmessage\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   1309\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1310\u001b[0m   \u001b[1;32mdef\u001b[0m \u001b[0m_extend_graph\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mFailedPreconditionError\u001b[0m: Attempting to use uninitialized value v2\n\t [[{{node v2/read}} = Identity[T=DT_FLOAT, _device=\"/job:localhost/replica:0/task:0/device:GPU:0\"](v2)]]\n\t [[{{node add/_3}} = _Recv[client_terminated=false, recv_device=\"/job:localhost/replica:0/task:0/device:CPU:0\", send_device=\"/job:localhost/replica:0/task:0/device:GPU:0\", send_device_incarnation=1, tensor_name=\"edge_9_add\", tensor_type=DT_FLOAT, _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"]()]]\n\nCaused by op 'v2/read', defined at:\n  File \"d:\\python3\\Lib\\runpy.py\", line 193, in _run_module_as_main\n    \"__main__\", mod_spec)\n  File \"d:\\python3\\Lib\\runpy.py\", line 85, in _run_code\n    exec(code, run_globals)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel_launcher.py\", line 16, in <module>\n    app.launch_new_instance()\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\traitlets\\config\\application.py\", line 658, in launch_instance\n    app.start()\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel\\kernelapp.py\", line 486, in start\n    self.io_loop.start()\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tornado\\platform\\asyncio.py\", line 112, in start\n    self.asyncio_loop.run_forever()\n  File \"d:\\python3\\Lib\\asyncio\\base_events.py\", line 421, in run_forever\n    self._run_once()\n  File \"d:\\python3\\Lib\\asyncio\\base_events.py\", line 1431, in _run_once\n    handle._run()\n  File \"d:\\python3\\Lib\\asyncio\\events.py\", line 145, in _run\n    self._callback(*self._args)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tornado\\platform\\asyncio.py\", line 102, in _handle_events\n    handler_func(fileobj, events)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tornado\\stack_context.py\", line 276, in null_wrapper\n    return fn(*args, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\zmq\\eventloop\\zmqstream.py\", line 450, in _handle_events\n    self._handle_recv()\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\zmq\\eventloop\\zmqstream.py\", line 480, in _handle_recv\n    self._run_callback(callback, msg)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\zmq\\eventloop\\zmqstream.py\", line 432, in _run_callback\n    callback(*args, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tornado\\stack_context.py\", line 276, in null_wrapper\n    return fn(*args, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel\\kernelbase.py\", line 283, in dispatcher\n    return self.dispatch_shell(stream, msg)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel\\kernelbase.py\", line 233, in dispatch_shell\n    handler(stream, idents, msg)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel\\kernelbase.py\", line 399, in execute_request\n    user_expressions, allow_stdin)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel\\ipkernel.py\", line 208, in do_execute\n    res = shell.run_cell(code, store_history=store_history, silent=silent)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel\\zmqshell.py\", line 537, in run_cell\n    return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2728, in run_cell\n    interactivity=interactivity, compiler=compiler, result=result)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2850, in run_ast_nodes\n    if self.run_code(code, result):\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2910, in run_code\n    exec(code_obj, self.user_global_ns, self.user_ns)\n  File \"<ipython-input-1-9ee245d85979>\", line 5, in <module>\n    v2 = tf.Variable(tf.constant(1.0, shape=[1]), name='v2')\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\ops\\variables.py\", line 145, in __call__\n    return cls._variable_call(*args, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\ops\\variables.py\", line 141, in _variable_call\n    aggregation=aggregation)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\ops\\variables.py\", line 120, in <lambda>\n    previous_getter = lambda **kwargs: default_variable_creator(None, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\ops\\variable_scope.py\", line 2441, in default_variable_creator\n    expected_shape=expected_shape, import_scope=import_scope)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\ops\\variables.py\", line 147, in __call__\n    return super(VariableMetaclass, cls).__call__(*args, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\ops\\variables.py\", line 1104, in __init__\n    constraint=constraint)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\ops\\variables.py\", line 1266, in _init_from_args\n    self._snapshot = array_ops.identity(self._variable, name=\"read\")\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\ops\\array_ops.py\", line 81, in identity\n    return gen_array_ops.identity(input, name=name)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\ops\\gen_array_ops.py\", line 3994, in identity\n    \"Identity\", input=input, name=name)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\framework\\op_def_library.py\", line 787, in _apply_op_helper\n    op_def=op_def)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\util\\deprecation.py\", line 488, in new_func\n    return func(*args, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\framework\\ops.py\", line 3272, in create_op\n    op_def=op_def)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\framework\\ops.py\", line 1768, in __init__\n    self._traceback = tf_stack.extract_stack()\n\nFailedPreconditionError (see above for traceback): Attempting to use uninitialized value v2\n\t [[{{node v2/read}} = Identity[T=DT_FLOAT, _device=\"/job:localhost/replica:0/task:0/device:GPU:0\"](v2)]]\n\t [[{{node add/_3}} = _Recv[client_terminated=false, recv_device=\"/job:localhost/replica:0/task:0/device:CPU:0\", send_device=\"/job:localhost/replica:0/task:0/device:GPU:0\", send_device_incarnation=1, tensor_name=\"edge_9_add\", tensor_type=DT_FLOAT, _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"]()]]\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "# 使用和保存模型代码中一样的方式来声明变量\n",
    "v1 = tf.Variable(tf.constant(1.0, shape=[1]), name='v1')\n",
    "v2 = tf.Variable(tf.constant(1.0, shape=[1]), name='v2')\n",
    "result = v1 + v2\n",
    "\n",
    "saver = tf.train.Saver([v1])\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    # 加载已经保存的模型，并通过已保存的模型中变量来计算加法\n",
    "    saver.restore(sess, \"Saved_model/model.ckpt\")\n",
    "    print(sess.run(result))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**4. 变量重命名**\n",
    "\n",
    "除了可以选取需要被保存或加载的变量，**`tf.train.Saver` 类也支持再保存或加载时给变量重命名**。如下面的程序中对变量v1和v2的名称进行了修改，如果直接通过 `tf.train.Saver` 默认的构造函数来加载保存的模型，那么程序会报变量找不到的错误 `NotFoundError: Key other-v1 not found in checkpoint`，如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from Saved_model/model.ckpt\n"
     ]
    },
    {
     "ename": "NotFoundError",
     "evalue": "Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:\n\nKey other-v1 not found in checkpoint\n\t [[{{node save/RestoreV2}} = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT], _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]\n\nCaused by op 'save/RestoreV2', defined at:\n  File \"d:\\python3\\Lib\\runpy.py\", line 193, in _run_module_as_main\n    \"__main__\", mod_spec)\n  File \"d:\\python3\\Lib\\runpy.py\", line 85, in _run_code\n    exec(code, run_globals)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel_launcher.py\", line 16, in <module>\n    app.launch_new_instance()\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\traitlets\\config\\application.py\", line 658, in launch_instance\n    app.start()\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel\\kernelapp.py\", line 486, in start\n    self.io_loop.start()\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tornado\\platform\\asyncio.py\", line 112, in start\n    self.asyncio_loop.run_forever()\n  File \"d:\\python3\\Lib\\asyncio\\base_events.py\", line 421, in run_forever\n    self._run_once()\n  File \"d:\\python3\\Lib\\asyncio\\base_events.py\", line 1431, in _run_once\n    handle._run()\n  File \"d:\\python3\\Lib\\asyncio\\events.py\", line 145, in _run\n    self._callback(*self._args)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tornado\\platform\\asyncio.py\", line 102, in _handle_events\n    handler_func(fileobj, events)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tornado\\stack_context.py\", line 276, in null_wrapper\n    return fn(*args, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\zmq\\eventloop\\zmqstream.py\", line 450, in _handle_events\n    self._handle_recv()\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\zmq\\eventloop\\zmqstream.py\", line 480, in _handle_recv\n    self._run_callback(callback, msg)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\zmq\\eventloop\\zmqstream.py\", line 432, in _run_callback\n    callback(*args, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tornado\\stack_context.py\", line 276, in null_wrapper\n    return fn(*args, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel\\kernelbase.py\", line 283, in dispatcher\n    return self.dispatch_shell(stream, msg)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel\\kernelbase.py\", line 233, in dispatch_shell\n    handler(stream, idents, msg)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel\\kernelbase.py\", line 399, in execute_request\n    user_expressions, allow_stdin)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel\\ipkernel.py\", line 208, in do_execute\n    res = shell.run_cell(code, store_history=store_history, silent=silent)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel\\zmqshell.py\", line 537, in run_cell\n    return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2728, in run_cell\n    interactivity=interactivity, compiler=compiler, result=result)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2850, in run_ast_nodes\n    if self.run_code(code, result):\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2910, in run_code\n    exec(code_obj, self.user_global_ns, self.user_ns)\n  File \"<ipython-input-1-732e38388eb2>\", line 8, in <module>\n    saver = tf.train.Saver()\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\training\\saver.py\", line 1094, in __init__\n    self.build()\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\training\\saver.py\", line 1106, in build\n    self._build(self._filename, build_save=True, build_restore=True)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\training\\saver.py\", line 1143, in _build\n    build_save=build_save, build_restore=build_restore)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\training\\saver.py\", line 787, in _build_internal\n    restore_sequentially, reshape)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\training\\saver.py\", line 406, in _AddRestoreOps\n    restore_sequentially)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\training\\saver.py\", line 854, in bulk_restore\n    return io_ops.restore_v2(filename_tensor, names, slices, dtypes)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\ops\\gen_io_ops.py\", line 1550, in restore_v2\n    shape_and_slices=shape_and_slices, dtypes=dtypes, name=name)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\framework\\op_def_library.py\", line 787, in _apply_op_helper\n    op_def=op_def)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\util\\deprecation.py\", line 488, in new_func\n    return func(*args, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\framework\\ops.py\", line 3272, in create_op\n    op_def=op_def)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\framework\\ops.py\", line 1768, in __init__\n    self._traceback = tf_stack.extract_stack()\n\nNotFoundError (see above for traceback): Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:\n\nKey other-v1 not found in checkpoint\n\t [[{{node save/RestoreV2}} = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT], _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]\n",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mNotFoundError\u001b[0m                             Traceback (most recent call last)",
      "\u001b[1;32md:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36m_do_call\u001b[1;34m(self, fn, *args)\u001b[0m\n\u001b[0;32m   1291\u001b[0m     \u001b[1;32mtry\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1292\u001b[1;33m       \u001b[1;32mreturn\u001b[0m \u001b[0mfn\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m*\u001b[0m\u001b[0margs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   1293\u001b[0m     \u001b[1;32mexcept\u001b[0m \u001b[0merrors\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mOpError\u001b[0m \u001b[1;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32md:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36m_run_fn\u001b[1;34m(feed_dict, fetch_list, target_list, options, run_metadata)\u001b[0m\n\u001b[0;32m   1276\u001b[0m       return self._call_tf_sessionrun(\n\u001b[1;32m-> 1277\u001b[1;33m           options, feed_dict, fetch_list, target_list, run_metadata)\n\u001b[0m\u001b[0;32m   1278\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32md:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36m_call_tf_sessionrun\u001b[1;34m(self, options, feed_dict, fetch_list, target_list, run_metadata)\u001b[0m\n\u001b[0;32m   1366\u001b[0m         \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_session\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0moptions\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfeed_dict\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfetch_list\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtarget_list\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1367\u001b[1;33m         run_metadata)\n\u001b[0m\u001b[0;32m   1368\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mNotFoundError\u001b[0m: Key other-v1 not found in checkpoint\n\t [[{{node save/RestoreV2}} = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT], _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]",
      "\nDuring handling of the above exception, another exception occurred:\n",
      "\u001b[1;31mNotFoundError\u001b[0m                             Traceback (most recent call last)",
      "\u001b[1;32md:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\training\\saver.py\u001b[0m in \u001b[0;36mrestore\u001b[1;34m(self, sess, save_path)\u001b[0m\n\u001b[0;32m   1537\u001b[0m         sess.run(self.saver_def.restore_op_name,\n\u001b[1;32m-> 1538\u001b[1;33m                  {self.saver_def.filename_tensor_name: save_path})\n\u001b[0m\u001b[0;32m   1539\u001b[0m     \u001b[1;32mexcept\u001b[0m \u001b[0merrors\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mNotFoundError\u001b[0m \u001b[1;32mas\u001b[0m \u001b[0merr\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32md:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36mrun\u001b[1;34m(self, fetches, feed_dict, options, run_metadata)\u001b[0m\n\u001b[0;32m    886\u001b[0m       result = self._run(None, fetches, feed_dict, options_ptr,\n\u001b[1;32m--> 887\u001b[1;33m                          run_metadata_ptr)\n\u001b[0m\u001b[0;32m    888\u001b[0m       \u001b[1;32mif\u001b[0m \u001b[0mrun_metadata\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32md:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36m_run\u001b[1;34m(self, handle, fetches, feed_dict, options, run_metadata)\u001b[0m\n\u001b[0;32m   1109\u001b[0m       results = self._do_run(handle, final_targets, final_fetches,\n\u001b[1;32m-> 1110\u001b[1;33m                              feed_dict_tensor, options, run_metadata)\n\u001b[0m\u001b[0;32m   1111\u001b[0m     \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32md:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36m_do_run\u001b[1;34m(self, handle, target_list, fetch_list, feed_dict, options, run_metadata)\u001b[0m\n\u001b[0;32m   1285\u001b[0m       return self._do_call(_run_fn, feeds, fetches, targets, options,\n\u001b[1;32m-> 1286\u001b[1;33m                            run_metadata)\n\u001b[0m\u001b[0;32m   1287\u001b[0m     \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32md:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36m_do_call\u001b[1;34m(self, fn, *args)\u001b[0m\n\u001b[0;32m   1307\u001b[0m         \u001b[0mmessage\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0merror_interpolation\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0minterpolate\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mmessage\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_graph\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1308\u001b[1;33m       \u001b[1;32mraise\u001b[0m \u001b[0mtype\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0me\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnode_def\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mop\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmessage\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   1309\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mNotFoundError\u001b[0m: Key other-v1 not found in checkpoint\n\t [[{{node save/RestoreV2}} = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT], _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]\n\nCaused by op 'save/RestoreV2', defined at:\n  File \"d:\\python3\\Lib\\runpy.py\", line 193, in _run_module_as_main\n    \"__main__\", mod_spec)\n  File \"d:\\python3\\Lib\\runpy.py\", line 85, in _run_code\n    exec(code, run_globals)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel_launcher.py\", line 16, in <module>\n    app.launch_new_instance()\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\traitlets\\config\\application.py\", line 658, in launch_instance\n    app.start()\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel\\kernelapp.py\", line 486, in start\n    self.io_loop.start()\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tornado\\platform\\asyncio.py\", line 112, in start\n    self.asyncio_loop.run_forever()\n  File \"d:\\python3\\Lib\\asyncio\\base_events.py\", line 421, in run_forever\n    self._run_once()\n  File \"d:\\python3\\Lib\\asyncio\\base_events.py\", line 1431, in _run_once\n    handle._run()\n  File \"d:\\python3\\Lib\\asyncio\\events.py\", line 145, in _run\n    self._callback(*self._args)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tornado\\platform\\asyncio.py\", line 102, in _handle_events\n    handler_func(fileobj, events)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tornado\\stack_context.py\", line 276, in null_wrapper\n    return fn(*args, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\zmq\\eventloop\\zmqstream.py\", line 450, in _handle_events\n    self._handle_recv()\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\zmq\\eventloop\\zmqstream.py\", line 480, in _handle_recv\n    self._run_callback(callback, msg)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\zmq\\eventloop\\zmqstream.py\", line 432, in _run_callback\n    callback(*args, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tornado\\stack_context.py\", line 276, in null_wrapper\n    return fn(*args, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel\\kernelbase.py\", line 283, in dispatcher\n    return self.dispatch_shell(stream, msg)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel\\kernelbase.py\", line 233, in dispatch_shell\n    handler(stream, idents, msg)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel\\kernelbase.py\", line 399, in execute_request\n    user_expressions, allow_stdin)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel\\ipkernel.py\", line 208, in do_execute\n    res = shell.run_cell(code, store_history=store_history, silent=silent)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel\\zmqshell.py\", line 537, in run_cell\n    return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2728, in run_cell\n    interactivity=interactivity, compiler=compiler, result=result)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2850, in run_ast_nodes\n    if self.run_code(code, result):\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2910, in run_code\n    exec(code_obj, self.user_global_ns, self.user_ns)\n  File \"<ipython-input-1-732e38388eb2>\", line 8, in <module>\n    saver = tf.train.Saver()\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\training\\saver.py\", line 1094, in __init__\n    self.build()\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\training\\saver.py\", line 1106, in build\n    self._build(self._filename, build_save=True, build_restore=True)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\training\\saver.py\", line 1143, in _build\n    build_save=build_save, build_restore=build_restore)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\training\\saver.py\", line 787, in _build_internal\n    restore_sequentially, reshape)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\training\\saver.py\", line 406, in _AddRestoreOps\n    restore_sequentially)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\training\\saver.py\", line 854, in bulk_restore\n    return io_ops.restore_v2(filename_tensor, names, slices, dtypes)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\ops\\gen_io_ops.py\", line 1550, in restore_v2\n    shape_and_slices=shape_and_slices, dtypes=dtypes, name=name)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\framework\\op_def_library.py\", line 787, in _apply_op_helper\n    op_def=op_def)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\util\\deprecation.py\", line 488, in new_func\n    return func(*args, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\framework\\ops.py\", line 3272, in create_op\n    op_def=op_def)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\framework\\ops.py\", line 1768, in __init__\n    self._traceback = tf_stack.extract_stack()\n\nNotFoundError (see above for traceback): Key other-v1 not found in checkpoint\n\t [[{{node save/RestoreV2}} = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT], _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]\n",
      "\nDuring handling of the above exception, another exception occurred:\n",
      "\u001b[1;31mNotFoundError\u001b[0m                             Traceback (most recent call last)",
      "\u001b[1;32md:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\training\\saver.py\u001b[0m in \u001b[0;36mrestore\u001b[1;34m(self, sess, save_path)\u001b[0m\n\u001b[0;32m   1547\u001b[0m       \u001b[1;32mtry\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1548\u001b[1;33m         \u001b[0mnames_to_keys\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mobject_graph_key_mapping\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0msave_path\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   1549\u001b[0m       \u001b[1;32mexcept\u001b[0m \u001b[0merrors\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mNotFoundError\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32md:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\training\\saver.py\u001b[0m in \u001b[0;36mobject_graph_key_mapping\u001b[1;34m(checkpoint_path)\u001b[0m\n\u001b[0;32m   1821\u001b[0m   object_graph_string = reader.get_tensor(\n\u001b[1;32m-> 1822\u001b[1;33m       checkpointable.OBJECT_GRAPH_PROTO_KEY)\n\u001b[0m\u001b[0;32m   1823\u001b[0m   object_graph_proto = (\n",
      "\u001b[1;32md:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\pywrap_tensorflow_internal.py\u001b[0m in \u001b[0;36mget_tensor\u001b[1;34m(self, tensor_str)\u001b[0m\n\u001b[0;32m    358\u001b[0m         return CheckpointReader_GetTensor(self, compat.as_bytes(tensor_str),\n\u001b[1;32m--> 359\u001b[1;33m                                           status)\n\u001b[0m\u001b[0;32m    360\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32md:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\framework\\errors_impl.py\u001b[0m in \u001b[0;36m__exit__\u001b[1;34m(self, type_arg, value_arg, traceback_arg)\u001b[0m\n\u001b[0;32m    525\u001b[0m             \u001b[0mcompat\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mas_text\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mc_api\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mTF_Message\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mstatus\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mstatus\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 526\u001b[1;33m             c_api.TF_GetCode(self.status.status))\n\u001b[0m\u001b[0;32m    527\u001b[0m     \u001b[1;31m# Delete the underlying status object from memory otherwise it stays alive\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mNotFoundError\u001b[0m: Key _CHECKPOINTABLE_OBJECT_GRAPH not found in checkpoint",
      "\nDuring handling of the above exception, another exception occurred:\n",
      "\u001b[1;31mNotFoundError\u001b[0m                             Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-1-732e38388eb2>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[0;32m     10\u001b[0m \u001b[1;32mwith\u001b[0m \u001b[0mtf\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mSession\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;32mas\u001b[0m \u001b[0msess\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     11\u001b[0m     \u001b[1;31m# 加载已经保存的模型，并通过已保存的模型中变量来计算加法\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 12\u001b[1;33m     \u001b[0msaver\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mrestore\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0msess\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m\"Saved_model/model.ckpt\"\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     13\u001b[0m     \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0msess\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mrun\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mresult\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32md:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\training\\saver.py\u001b[0m in \u001b[0;36mrestore\u001b[1;34m(self, sess, save_path)\u001b[0m\n\u001b[0;32m   1552\u001b[0m         \u001b[1;31m# a helpful message (b/110263146)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1553\u001b[0m         raise _wrap_restore_error_with_msg(\n\u001b[1;32m-> 1554\u001b[1;33m             err, \"a Variable name or other graph key that is missing\")\n\u001b[0m\u001b[0;32m   1555\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1556\u001b[0m       \u001b[1;31m# This is an object-based checkpoint. We'll print a warning and then do\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mNotFoundError\u001b[0m: Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:\n\nKey other-v1 not found in checkpoint\n\t [[{{node save/RestoreV2}} = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT], _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]\n\nCaused by op 'save/RestoreV2', defined at:\n  File \"d:\\python3\\Lib\\runpy.py\", line 193, in _run_module_as_main\n    \"__main__\", mod_spec)\n  File \"d:\\python3\\Lib\\runpy.py\", line 85, in _run_code\n    exec(code, run_globals)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel_launcher.py\", line 16, in <module>\n    app.launch_new_instance()\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\traitlets\\config\\application.py\", line 658, in launch_instance\n    app.start()\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel\\kernelapp.py\", line 486, in start\n    self.io_loop.start()\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tornado\\platform\\asyncio.py\", line 112, in start\n    self.asyncio_loop.run_forever()\n  File \"d:\\python3\\Lib\\asyncio\\base_events.py\", line 421, in run_forever\n    self._run_once()\n  File \"d:\\python3\\Lib\\asyncio\\base_events.py\", line 1431, in _run_once\n    handle._run()\n  File \"d:\\python3\\Lib\\asyncio\\events.py\", line 145, in _run\n    self._callback(*self._args)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tornado\\platform\\asyncio.py\", line 102, in _handle_events\n    handler_func(fileobj, events)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tornado\\stack_context.py\", line 276, in null_wrapper\n    return fn(*args, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\zmq\\eventloop\\zmqstream.py\", line 450, in _handle_events\n    self._handle_recv()\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\zmq\\eventloop\\zmqstream.py\", line 480, in _handle_recv\n    self._run_callback(callback, msg)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\zmq\\eventloop\\zmqstream.py\", line 432, in _run_callback\n    callback(*args, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tornado\\stack_context.py\", line 276, in null_wrapper\n    return fn(*args, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel\\kernelbase.py\", line 283, in dispatcher\n    return self.dispatch_shell(stream, msg)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel\\kernelbase.py\", line 233, in dispatch_shell\n    handler(stream, idents, msg)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel\\kernelbase.py\", line 399, in execute_request\n    user_expressions, allow_stdin)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel\\ipkernel.py\", line 208, in do_execute\n    res = shell.run_cell(code, store_history=store_history, silent=silent)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\ipykernel\\zmqshell.py\", line 537, in run_cell\n    return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2728, in run_cell\n    interactivity=interactivity, compiler=compiler, result=result)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2850, in run_ast_nodes\n    if self.run_code(code, result):\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2910, in run_code\n    exec(code_obj, self.user_global_ns, self.user_ns)\n  File \"<ipython-input-1-732e38388eb2>\", line 8, in <module>\n    saver = tf.train.Saver()\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\training\\saver.py\", line 1094, in __init__\n    self.build()\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\training\\saver.py\", line 1106, in build\n    self._build(self._filename, build_save=True, build_restore=True)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\training\\saver.py\", line 1143, in _build\n    build_save=build_save, build_restore=build_restore)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\training\\saver.py\", line 787, in _build_internal\n    restore_sequentially, reshape)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\training\\saver.py\", line 406, in _AddRestoreOps\n    restore_sequentially)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\training\\saver.py\", line 854, in bulk_restore\n    return io_ops.restore_v2(filename_tensor, names, slices, dtypes)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\ops\\gen_io_ops.py\", line 1550, in restore_v2\n    shape_and_slices=shape_and_slices, dtypes=dtypes, name=name)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\framework\\op_def_library.py\", line 787, in _apply_op_helper\n    op_def=op_def)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\util\\deprecation.py\", line 488, in new_func\n    return func(*args, **kwargs)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\framework\\ops.py\", line 3272, in create_op\n    op_def=op_def)\n  File \"d:\\python3\\tfgpu\\dl+\\lib\\site-packages\\tensorflow\\python\\framework\\ops.py\", line 1768, in __init__\n    self._traceback = tf_stack.extract_stack()\n\nNotFoundError (see above for traceback): Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:\n\nKey other-v1 not found in checkpoint\n\t [[{{node save/RestoreV2}} = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT], _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "# 注意定义时的name不一样！！！\n",
    "v1 = tf.Variable(tf.constant(1.0, shape=[1]), name=\"other-v1\")\n",
    "v2 = tf.Variable(tf.constant(1.0, shape=[1]), name=\"other-v2\")\n",
    "result = v1 + v2\n",
    "\n",
    "saver = tf.train.Saver()\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    # 加载已经保存的模型，并通过已保存的模型中变量来计算加法\n",
    "    saver.restore(sess, \"Saved_model/model.ckpt\")\n",
    "    print(sess.run(result))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这是因为模型保存时的变量名和加载时的变量名不同，为了解决这个问题，**TensorFlow可以通过字典将模型保存时的变量名和需加载的变量联系起来**，如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from Saved_model/model.ckpt\n",
      "[2.]\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "# 注意定义时的name不一样！！！\n",
    "v1 = tf.Variable(tf.constant(1.0, shape=[1]), name=\"other-v1\")\n",
    "v2 = tf.Variable(tf.constant(1.0, shape=[1]), name=\"other-v2\")\n",
    "result = v1 + v2\n",
    "\n",
    "# 将原来名称为v1的变量加载到现在变量v1中（名称为other-v1）,将原来名称为v2的变量加载到现在变量v1中（名称为other-v2）\n",
    "saver = tf.train.Saver({\"v1\": v1, \"v2\": v2})\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    # 加载已经保存的模型，并通过已保存的模型中变量来计算加法\n",
    "    saver.restore(sess, \"Saved_model/model.ckpt\")\n",
    "    print(sess.run(result))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "**5. 滑动平均类的保存和加载**\n",
    "\n",
    "支持变量重命名的主要目的之一就是方便使用变量的滑动平均值。4.4.3节中介绍了使用滑动平均值可以让模型更加健壮，在TensorFlow中每个变量的滑动平均值是通过影子变量维护的，所以要获取变量的滑动平均值实际上就是获取这个影子变量的取值。**如果在加载加载模型时，直接将影子变量映射到变量自身，那么使用训练好的模型时就不需要调用函数来获取变量的滑动平均值了**。下面给出了一个样例（再次提醒，这里需要restart the kernel，且从这开始运行cell）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "v:0\n",
      "v:0\n",
      "v/ExponentialMovingAverage:0\n",
      "[10.0, 0.099999905]\n"
     ]
    }
   ],
   "source": [
    "# 保存滑动平均模型\n",
    "import tensorflow as tf\n",
    "\n",
    "v = tf.Variable(0, dtype=tf.float32, name=\"v\")\n",
    "# 在没有申明滑动平均模型时只有一个变量v，所以以下语句只会输出\"v:0\"\n",
    "for variables in tf.global_variables():\n",
    "    print(variables.name)\n",
    "    \n",
    "ema = tf.train.ExponentialMovingAverage(0.99)\n",
    "maintain_averages_op = ema.apply(tf.global_variables())\n",
    "# 在申明滑动平均模型之后，TensorFlow会自动生成一个影子变量，于是以下语句会输出\"v:0\"和\"v/ExponentialMovingAverage:0\"\n",
    "for variables in tf.global_variables(): \n",
    "    print(variables.name)\n",
    "    \n",
    "saver = tf.train.Saver()\n",
    "with tf.Session() as sess:\n",
    "    init_op = tf.global_variables_initializer().run()\n",
    "    \n",
    "    sess.run(tf.assign(v, 10))\n",
    "    sess.run(maintain_averages_op)\n",
    "    # 保存的时候会将 v:0 和 v/ExponentialMovingAverage:0 这两个变量都存下来\n",
    "    saver.save(sess, \"Saved_model/model2.ckpt\")\n",
    "    print(sess.run([v, ema.average(v)]))     # 输出： [10.0, 0.099999905]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from Saved_model/model2.ckpt\n",
      "0.099999905\n"
     ]
    }
   ],
   "source": [
    "# 加载滑动平均模型，这个cell不用restart the kernel\n",
    "v = tf.Variable(0, dtype=tf.float32, name=\"v\")\n",
    "\n",
    "# 通过变量重命名将原来变量v的滑动平均值直接赋值给v。\n",
    "saver = tf.train.Saver({\"v/ExponentialMovingAverage\": v})\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    saver.restore(sess, \"Saved_model/model2.ckpt\")\n",
    "    print(sess.run(v))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了方便加载重命名滑动平均变量，`tf.train.ExponentialMovingAverage` 类提供了 `variables_to_restore` 函数来生成 `tf.train.Saver` 类所需要的变量重命名字典，如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'v/ExponentialMovingAverage': <tf.Variable 'v:0' shape=() dtype=float32_ref>}\n",
      "INFO:tensorflow:Restoring parameters from Saved_model/model2.ckpt\n",
      "0.099999905\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "v = tf.Variable(0, dtype=tf.float32, name=\"v\")\n",
    "\n",
    "ema = tf.train.ExponentialMovingAverage(0.99)\n",
    "\n",
    "# 通过variables_to_restore函数可以直接生成上面代码中提供的字典：{\"v/ExponentialMovingAverage\": v}\n",
    "print(ema.variables_to_restore())     # 输出：{'v/ExponentialMovingAverage': <tf.Variable 'v:0' shape=() dtype=float32_ref>}\n",
    "\n",
    "saver = tf.train.Saver(ema.variables_to_restore())\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    saver.restore(sess, \"Saved_model/model2.ckpt\")\n",
    "    print(sess.run(v))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**6. 常量方式保存（pb格式）**\n",
    "\n",
    "**使用 `tf.train.Saver` 会保存运行TensorFlow程序所需要的全部信息，然而有时并不需要某些信息**。比如在测试或者离线预测时，只需要知道如何从神经网络的输入层经过前向传播计算得到输出层即可，而不需要类似于变量初始化、模型保存等辅助节点的信息，在第6章介绍迁移学习时会遇到类似的情况。而且，**将变量取值和计算图结构分成不同的文件存储有时候也不方便**。\n",
    "\n",
    "于是TensorFlow提供了 `convert_variables_to_constants` 函数，通过这个函数可以将计算图中的变量及其取值通过常量的方式保存，这样整个TensorFlow计算图可以统一存放在一个文件中。**注意在保存选择节点时是['add']，即计算节点的名称，而在加载时是['add:0']，即张量的名称**。下面给出一个样例："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Froze 2 variables.\n",
      "INFO:tensorflow:Converted 2 variables to const ops.\n"
     ]
    }
   ],
   "source": [
    "# 保存\n",
    "import tensorflow as tf\n",
    "from tensorflow.python.framework import graph_util\n",
    "\n",
    "v1 = tf.Variable(tf.constant(1.0, shape=[1]), name=\"v1\")\n",
    "v2 = tf.Variable(tf.constant(2.0, shape=[1]), name=\"v2\")\n",
    "result = v1 + v2\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    init_op = tf.global_variables_initializer()\n",
    "    sess.run(init_op)\n",
    "    \n",
    "    # 导出当前计算图的GraphDef部分，只需要这一部分就可以完成从输入层到输出层的计算过程\n",
    "    graph_def = tf.get_default_graph().as_graph_def()\n",
    "    \n",
    "    # 将圈中的变量及其取值转化为常量，同时将图中不必要的节点去掉。在5.4.2 节中将会看\n",
    "    # 到一些系统运算也会被转化为计算图中的节点（比如变量初始化操作）。如果只关心程序中定\n",
    "    # 义的某些计算时，和这些计算无关的节点就没有必要导出并保存了。在下面一行代码中，最\n",
    "    # 后一个参数［'add'］给出了需要保存的节点名称。add 节点是上面定义的两个变量相加的\n",
    "    # 操作。注意这里给出的是计算节点的名称，所以没有后面的：0 。\n",
    "    output_graph_def = graph_util.convert_variables_to_constants(sess, graph_def, ['add'])\n",
    "    \n",
    "    with tf.gfile.GFile(\"Saved_model/combined_model.pb\", \"wb\") as f:\n",
    "           f.write(output_graph_def.SerializeToString())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[array([3.], dtype=float32)]\n"
     ]
    }
   ],
   "source": [
    "# 加载，这个cell不用restart the kernel\n",
    "from tensorflow.python.platform import gfile\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    model_filename = \"Saved_model/combined_model.pb\"\n",
    "    \n",
    "    # 读取保存的模型文件，并将文件解析成对应的GraphDef Protocol Buffer\n",
    "    with gfile.FastGFile(model_filename, 'rb') as f:\n",
    "        graph_def = tf.GraphDef()\n",
    "        graph_def.ParseFromString(f.read())\n",
    "\n",
    "    # 将graph_def中保存的图加载到当前的图中。return_elements＝[\"add:0\"]给出了返回的张量\n",
    "    # 的名称。在保存的时候给出的是计算节点的名称，所以为\"add\"，在加载的时候给出\n",
    "    # 的是张量的名称，所以为\"add:0\"。\n",
    "    result = tf.import_graph_def(graph_def, return_elements=[\"add:0\"])\n",
    "    print(sess.run(result))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
